I could not find a declareModule invocation in stylesheet's __init__.ily file in this branch. Anyway I tried the following for GridLY and it works fine

\declareLibrary "GridLY" \with {
  maintainers = #'("Matteo Ceccarello <[email protected]>")
  version = "0.4.0"
  short-description = \markup {
    GridLY is a library that implements a simple "segmented grid" approach
  }
}

is this the intended way of using it? I like this approach, I think this is the right direction, since it allows for more checks than simple comments.

Maybe it's a little too early to have the full benefits of explicit versioning (like for expressing dependencies between libraries). However, I think that declaring the version is useful because:

1. I allows us to adopt a meaningful versioning scheme, that can be used to communicate (potentially breaking) changes to the users. An example is Semantic Versioning
2. A user declaring a particular version of the software in his library can get warnings if updating the openLilyLib repository results in a newer version of a library that might introduce breaking changes. Consider the following example, using semantic versioning. Someone has a score that depends on GridLY 1.4.5 and updates the OLL repository to get the latest version of, say, ScholarLY. I can see these four scenarios for the dependency on GridLY:
   • The version of GridLY did not change with the pull, the code works as before.
   • There is a patch update, like 1.4.6. This means that it's an update that shouldn't affect user's code, so no warning is issued.
   • There is a minor update, like 1.5.0. This means that there are non-breaking changes but the user should pay attention, so a warning is printed as soon as the library is loaded.
   • There is a major update, say 2.0.0. Since this advertises breaking API changes, user's code may need to be changed. Throwing an error here can be appropriate, so that the user directly sees the source of the problem (the version change) rather than an unsorted mess of errors due to changes in the API.
3. It's future proof. If some day we'll go for something like a package manager, the infrastructure will be already in place.

Anyway, for me your implementation is OK, but since this is a fundamental change in the infrastructure I think that it would be good to hear the opinion of the others too.

Oh, I realized too late that I hadn't pushed everything, sorry.

I continued today and went in a very promising direction. But in the end I ran into very strange issues with regard to including files through ly:parser-include-string and didn't succeed so far.

I'll come back to this with comments and pushes later tonight.

is this the intended way of using it? I like this approach, I think this is the right direction, since it allows for more checks than simple comments.

Yes, that is about the way \declareLibrary? is intended.

• For maintainers you can use a simple string when there's only one.
• Note that version is currently explicitly restricted to 3-element lists. So `version = "0.3" would cause an error ATM.
  This is something that might be discussed.
• I have added a mandatory description option, and I have changed both description fields to string?
  This may also be discussed, the problem is that I don't really know if these fields would rather be used by parsing the file with Python to generate documentation, or if they are used with LilyPond as they are now to produce the usage examples.
  In a way I lean towards the Python/string solution as this is definitely a cleaner one. OTOH this might unnecessarily prevent usage in LilyPond scores.
• Additionally I have implemented the notion of "known options". These are not mandatory but if they are present they are also type-checked. Currently there are lilypond-min-version and lilypond-max-version.

I think the set of mandatory and recognized options should be designed so that it makes a kind of "introductory interface" to a library.

In addition to \useLibrary I have by now (that is, since the initial pull request) added \useModule so that \loadModule has now been completely deprecated. The main differences in the interface are the dot-list addressing of modules and the optional \with block.

A module can be addressed with a dot-path that is matched against the directory structure. The command looks for either .ily files or a subdirectory with a __main__.ily file. If also a __init__.ily this is loaded first.

This seems to work very well. But when trying to remove the remaining \loadModule invocations (on top of the latest pushed commit) I run into different sorts of strange errors. They range from surprising errors in Scheme functions over definitions that don't trigger warnings but that simply aren't present afterwards to strange errors about normal LilyPond code that is accused of being wrong.
I have the impression that there are some things going on with including files through (ly:parser-include-string, both using ly:gulp-file or using the "\\include " approach.

This is where I've currently got stuck with.

Regarding the comments about the versioning I fully agree. It's good to have it right from the start, but not trying to push it too far at the moment.

OK, I'm fine with all these changes, so once you solve that issue with loadModule invocation (I'll try to have a look at it) we can merge this. Anyway, as I've said, I'd like to know the opinion of the others too before the merge.

Just one observation:

I have changed both description fields to string?.
This may also be discussed, the problem is that I don't really know if these fields would rather be used by parsing the file with Python to generate documentation, or if they are used with LilyPond as they are now to produce the usage examples.
In a way I lean towards the Python/string solution as this is definitely a cleaner one. OTOH this might unnecessarily prevent usage in LilyPond scores.

I'd prefer the Python/string solution, since those fields will used for documentation on Python land for sure.

Am 15. März 2015 16:50:34 MEZ, schrieb Matteo Ceccarello [email protected]:

OK, I'm fine with all these changes, so once you solve that issue with
loadModule invocation (I'll try to have a look at it) we can merge
this. Anyway, as I've said, I'd like to know the opinion of the others
too before the merge.

U'm into something else right now but I'll ask more people about their opinion before merging.

Just one observation:

I have changed both description fields to string?.
This may also be discussed, the problem is that I don't really know if
these fields would rather be used by parsing the file with Python to
generate documentation, or if they are used with LilyPond as they are
now to produce the usage examples.
In a way I lean towards the Python/string solution as this is
definitely a cleaner one. OTOH this might unnecessarily prevent usage
in LilyPond scores.

I'd prefer the Python/string solution, since those fields will used for
documentation on Python land for sure.

Ok, sounds reasonable. That also opens up some Markdown options.

OK; I've added some more commits, updated the usage-examples and unit-tests to the new calling syntax and sorted several things out. I am still sitting on one annoying bug but have at least found a workaround so everything compiles for now.

Please @PaulMorris, @PeterBjuhr, @davidnalesnik, @janek-warchol and @jpvoigt have a look at this, giving me your opinion on the new interface, and possibly give me a helping hand with the bug.

For getting an idea about the interface you can read this Wiki page (only this one is up-to-date, particularly the "Common functionality" page would be important to update too) and iterate through the ly directory hierarchy, inspecting and compiling all .ly files in any usage-examples or unit-tests directory.

The bug is with the function \useModule (defined in ly/_internal/module-handling.ily). You'll notice that I've inserted useless #(display "") commands after each invocation. If you comment them there will be lots of different and strange errors, and my impression is that this has to do with the optional argument inside the command. But in fact I don't have the faintest clue about the reason.

Apart from this (critical) bug I'm very happy with the changes and the potential it has for simplifying our life with using extensions for LilyPond, so I would really like to be able to merge this ASAP.

I have narrowed it down somewhat more: The bug is related to the dot-list invocation and the list? predicate.

a) there are very different error messages depending on what comes next after the \useModule call:

• \markup something (like in scholarly/usage-examples/diplomatic-line-breaks.ly)
  ":1:1: error: unknown escaped string: \include' \include "/home/uliska/git/openlilylib/openlilylib/ly/scholarly/diplomatic-line-breaks.ily"" (this is from line 283 ofmodule-handling.ily`)
• a music expression like { c }
  "/home/uliska/git/openlilylib/openlilylib/ly/scholarly/diplomatic-line-breaks.ily:3:1: error: syntax error, unexpected \header"
  (this is the beginning of the included file)
• \relative { c }
  "/home/uliska/git/openlilylib/openlilylib/ly/scholarly/diplomatic-line-breaks.ily:3:1: error: syntax error, unexpected \header, expecting (backed-up?)"
• `#(display "")
  Everything fine

This already led me to think that the command somehow stumbles over the next item and can't distinguish it from an additional list element. But the proof is that when entering the argument in standard list notation

\useModule #'(scholarly diplomatic-line-breaks)
% instead of
\useModule scholarly.diplomatic-line-breaks

everything works as expected.

Unfortunately I can't easily reproduce this behaviour with a minimal example, but it has to be related to this dot-list parsing somehow.

Any ideas?

For example the following works as expected, and I can't see where \useModule has significant differences to its signature handling:

\version "2.19.16"

testDotList =
#(define-void-function (parser location opts path)
   ((ly:context-mod?) list?)
   (for-each
    (lambda (p)
      (display p)
      (newline))
    path))


\testDotList path.to.whatever

\testDotList \with {
  optionOne = ##t
  optionTwo = ##f
}
path.to.something.else

\relative { c' }